Inspection of systems and components in an industrial setting may involve some form of visual analysis. For example, quality inspection of manufactured items may detect anomalies or defects at an early stage of the production process. Current methods of anomaly detection include manual inspection by a specialist or machine vision based approaches to detect anomalies by processing captured images. Manual inspection is unsuitable for real time applications since it has low degree of automation and depends heavily on the experience of the specialist. Machine vision based approaches require strong supervision in the form of pixel-wise labeling (i.e., labeling each pixel of an image as either having anomaly or not) or bounding box annotation by outlining the anomalous parts of an image. These fully supervised methods are difficult to scale up and also time-consuming since they require a large amount of manual labeling in form of pixels or bounding box annotations.